Fully convolutional open set segmentation

In traditional semantic segmentation, knowing about all existing classes is essential to yield effective results with the majority of approaches. However, these methods trained in a Closed Set fail when new are found test phase, not being able recognize that an unseen class has been fed. This means they suitable for Open scenarios, which very common real-world computer vision and remote sensing applications. this paper, we discuss limitations segmentation propose two fully convolutional approaches effectively address segmentation: OpenFCN OpenPCS. based on well-known OpenMax algorithm, configuring application approach settings. OpenPCS novel feature-space from DNN activations serve as features computing PCA multi-variate gaussian likelihood lower dimensional space. addition aiming reduce RAM memory requirements methodology, also slight variation method (OpenIPCS) uses iteractive version be small batches. Experiments were conducted ISPRS Vaihingen/Potsdam 2018 IEEE GRSS Data Fusion Challenge datasets. showed little-to-no improvement compared simpler much more time efficient SoftMax thresholding, while some orders magnitude slower. achieved promising almost experiments by overcoming both thresholding. reasonable compromise between runtime performances extremely fast thresholding slow OpenFCN, run close real-time. indicate effective, robust improve recognition unknown pixels without reducing accuracy known pixels. We tested scenario hiding multiple simulate multimodal unknowns, resulting even larger gap OpenPCS/OpenIPCS implying modeling settings greater openness.